System and method for encoded spatio-spectral information processing

ABSTRACT

Encoded spatio-spectral information processing is performed using a system having a radiation source, wavelength dispersion device and two-dimensional switching array, such as digital micro-mirror array (DMA). In one aspect, spectral components from a sample are dispersed in space and modulated separately by the switching array, each element of which may operate according to a predetermined encoding pattern. The encoded spectral components can then be detected and analyzed. In a different aspect, the switching array can be used to provide a controllable radiation source for illuminating a sample with radiation patterns that have predetermined characteristics and separately encoded components. Various applications are disclosed.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a divisional of application Ser. No. 09/798,860,filed Mar. 1, 2001, which is a continuation-in-part of application Ser.No. 09/672,257, filed Sep. 28, 2000, which is a continuation ofapplication Ser. No. 09/502,758 filed February 11, 2000, now U.S. Pat.No. 6,128,078, which is a continuation of application Ser. No.09/289,482 filed Apr. 9, 1999, now U.S. Pat. No. 6,046,808. The contentof the above applications are incorporated herein by reference.

FIELD OF THE INVENTION

The present invention relates generally to signal processing, and moreparticularly to devices and methods for use in spectroscopy, imaging,spatial and spectral modulation filtering, controllable radiation sourcedesign and related signal processing.

BACKGROUND OF THE INVENTION

Imagers employ either a two-dimensional (2D) multichannel detector arrayor a single element detector. Imagers using a 2D detector array measurethe intensity distribution of all spatial resolution elementssimultaneously during the entire period of data acquisition. Imagersusing a single detector require that the individual spatial resolutionelements be measured consecutively via a raster scan so that each one isobserved for a small fraction of the period of data acquisition. Priorart imagers using a plurality of detectors at the image plane canexhibit serious signal-to-noise ratio problems. Prior art imagers usinga single element detector can exhibit more serious signal-to-noise ratioproblems. Signal-to-noise ratio problems limit the utility of imagersapplied to chemical imaging applications where subtle differencesbetween a sample's constituents become important.

Spectrometers are commonly used to analyze the chemical composition ofsamples by determining the absorption or attenuation of certainwavelengths of electromagnetic radiation by the sample or samples.Because it is typically necessary to analyze the absorptioncharacteristics of more than one wavelength of radiation to identify acompound, and because each wavelength must be separately detected todistinguish the wavelengths, prior art spectrometers utilize a pluralityof detectors, have a moving grating, or use a set of filter elements.However, the use of a plurality of detectors or the use of a macromoving grating has signal-to-noise limitations. The signal-to-noiseratio largely dictates the ability of the spectrometer to analyze withaccuracy all of the constituents of a sample, especially when some ofthe constituents of the sample account for an extremely small proportionof the sample. There is, therefore, a need for imagers and spectrometerswith improved signal-to-noise ratios.

Prior art variable band pass filter spectrometers, variable band rejectfilter spectrometers, variable multiple band pass filter spectrometersor variable multiple band reject filter spectrometers typically employ amultitude of filters that require macro moving parts or other physicalmanipulation in order to switch between individual filter elements orsets of filter elements for each measurement. Each filter elementemployed can be very expensive, difficult to manufacture and all arepermanently set at the time of manufacture in the wavelengths (bands) ofradiation that they pass or reject. Physical human handling of thefilter elements can damage them and it is time consuming to changefilter elements. There is, therefore, a need for variable band passfilter spectrometers, variable band reject filter spectrometers,variable multiple band pass filter spectrometers or variable multipleband reject filter spectrometers without a requirement for discrete(individual) filter elements that have permanently set band pass or bandreject properties. There is also a need for variable band pass filterspectrometers, variable band reject filter spectrometers, variablemultiple band pass filter spectrometers or variable multiple band rejectfilter spectrometers to be able to change the filters corresponding tothe bands of radiation that are passed or rejected rapidly, withoutmacro moving parts and without human interaction.

In several practical applications it is required that an object beirradiated with radiation having particularly shaped spectrum. In thesimplest case when only a few spectrum lines (or bands) are necessary,one can use a combination of corresponding sources, each centered near arequired spectrum band. Clearly, however, this approach does not work ina more general case, and therefore it is desirable to have acontrollable radiation source capable of providing arbitrary spectrumshapes and intensities. Several types of prior art devices are knownthat are capable of providing controllable radiation. Earlier prior artdevices primarily relied upon various “masking” techniques, such aselectronically alterable masks interposed in the optical pathway betweena light source and a detector. More recent prior art devices use acombination of two or more light-emitting diodes (LEDs) as radiationsources. In such cases, an array of LEDs or light-emitting lasers isconfigured for activation using a particular encoding pattern, and canbe used as a controllable light source. A disadvantage of these systemsis that they rely on an array of different LED elements (or lasers),each operating in a different, relatively narrow spectrum band. Inaddition, there are technological problems associated with having anarray of discrete radiation elements with different characteristics.Accordingly, there is a need for a controllable radiation source, wherevirtually arbitrary spectrum shape and characteristics can be designed,and where disadvantages associated with the prior art are obviated.Further, it is desirable not only to shape the spectrum of the radiationsource, but also encode its components differently, which feature can beused to readily perform several signal processing functions useful in anumber of practical applications. The phrase “a spectrum shape” in thisdisclosure refers not to a mathematical abstraction but rather toconfigurable spectrum shapes having range(s) and resolution necessarilylimited by practical considerations.

In addition to the signal-to-noise issues discussed above, one canconsider the tradeoff between signal-to-noise and, for example, one ormore of the following resources: system cost, time to measure a scene,and inter-pixel calibration. Thus, in certain prior art systems, asingle sensor system may cost less to produce, but will take longer tofully measure an object under study. In prior art multi-sensor systems,one often encounters a problem in which the different sensor elementshave different response characteristics, and it is necessary to addcomponents to the system to calibrate for this. It is desirable to havea system with which one gains the lower-cost, better signal-to-noise,and automatic inter-pixel calibration advantages of a single-sensorsystem, while not suffering all of the time loss usually associated withusing single sensors.

SUMMARY OF THE INVENTION

In one aspect, the present invention solves the above-described problemsand provides a distinct advance in the art by providing an imager orspectrometer that is less sensitive to ambient noise and that caneffectively operate even when used in environments with a high level ofambient radiation. The invention further advances the art of variableband pass filter spectrometers, variable band reject filterspectrometers, variable multiple band pass filter spectrometers orvariable multiple band reject filter spectrometers by providing avariable band pass filter spectrometer, variable band reject filterspectrometer, variable multiple band pass filter spectrometer orvariable multiple band reject filter spectrometer that: (1) does notrequire the selection of the bands of wavelengths passed or rejected atthe time of manufacture; (2) allows the selection of any desiredcombination of bands of wavelengths that are passed or rejected; (3)reduces the time to change the bands of wavelengths passed or rejected;and (4) requires no macro moving parts to accomplish a change in thebands of wavelengths passed or rejected.

In a first aspect, the system of the present invention generallyincludes one or more radiation sources, a two-dimensional array ofmodulateable micro-mirrors or an equivalent switching structure, adetector, and an analyzer. In a specific embodiment, the two-dimensionalswitching array is positioned for receiving an image. The micro-mirrors(or corresponding switching elements of the array) are modulated inorder to reflect individual spatially-distributed radiation componentsof the image toward the detector. In a preferred embodiment, themodulation is performed using known and selectively different modulationrates.

According to this aspect of the invention, a detector is oriented toreceive the combined radiation components reflected from the array andis operable to generate an output signal representative of the combinedradiation incident thereon. The analyzer is operably coupled with thedetector to receive the output signal and to demodulate the signal torecover signals representative of each of the individual spatiallydistributed radiation components of the image. The analyzer can beconfigured to recover all reflected components or to reject someunnecessary components of the recovered signals from the combinedreflections.

By using micro-mirrors that receive the individual spectral or spatialradiation components and then modulate these components at differentmodulation rates, all of the radiation components can be focused onto asingle detector and then demodulated to maximize the signal-to-noiseratio (SNR) of the detector. Various techniques for enhancing the SNR ofthe system are presented as well.

In another important aspect, the present invention provides a distinctadvance in the state of the art by enabling the design of a controllableradiation source, which uses no masking elements, which are generallyslow and cumbersome to operate, and no discrete light sources, whichalso present a number of technical issues in practice. Instead, thecontrollable radiation source in accordance with a preferred embodimentis implemented using a broadband source illuminating a two-dimensionalarray of switching elements, such as a DMA. Modulation of the individualswitching elements of the array provides an easy mechanism forspatio-spectral encoding of the input radiation, which encoding can beused in a number of practical applications.

In accordance with another aspect of the invention, a two-dimensionalarray of switching elements, such as a DMA, can be configured and usedas a basic building block for various optical processing tasks, and isreferred to as an optical synapse processing unit (OSPU). Combinationsof OSPUs with standard processing components can be used in thepreferred embodiments of the present invention in a number of practicalapplications, including data compression, feature extraction and others.In a specific embodiment, a spectrometer using a controlled radiationsource provides for very rapid analysis of a sample using an orthogonalset of basis functions, such as Hadamard or Fourier transformtechniques, resulting in significantly enhanced signal-to-noise ratio.

The present invention gains the lower-cost, better signal-to-noise, andautomatic inter-pixel calibration advantages of single-sensor systems,while not suffering all of the time loss usually associated with usingsingle sensors, because it allows for adaptive and tunable acquisitionof only the desired information, as opposed to prior-art systems whichare generally full data-cube acquisition devices requiring additionalpost processing to discover or recover the knowledge ultimately soughtin the application of the system.

One skilled in the art will recognize that, while the invention here isdescribed using 2D arrays of micro-mirrors, any 2D spatial lightmodulator can be used. It should also be noted that a pair, or a few 1Dspatial light modulators can be combined to effectively produce a 2Dspatial light modulator for applications that involve raster scanning,Walsh-Hadamard scanning, or scanning or acquisition with any separablelibrary of patterns.

It is intended that the devices and methods in this application ingeneral are capable of operating in various ranges of electromagneticradiation, including the ultraviolet, visible, infrared, and microwavespectrum portions. Further, it will be appreciated by those of skill inthe art of signal processing, be it acoustic, electric, magnetic, etc.,that the devices and techniques disclosed herein for optical signalprocessing can be applied in a straight-forward way to those othersignals as well.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be understood and appreciated more fully fromthe following detailed description, taken in conjunction with thedrawings in which:

FIGS. 1A and 1B are schematic diagrams illustrating a spectrometerconstructed in accordance with two embodiments of the invention;

FIG. 2 is a plan view of a micro-mirror array used in the presentinvention;

FIG. 3 is a schematic diagram of two micro-mirrors illustrating themodulations of the mirrors of the micro-mirror device of FIG. 2;

FIG. 4 is a graph illustrating an output signal of the spectrometer whenused to analyze the composition of a sample;

FIG. 5 is a graph illustrating an output signal of the imager when usedfor imaging purposes;

FIG. 6 is a schematic diagram illustrating an imager constructed inaccordance with a preferred embodiment of the invention; FIG. 6Aillustrates spatio-spectral distribution of a DMA, where individualelements can be modulated;

FIG. 7 is an illustration of the input to the DMA Filter Spectrometerand its use to pass or reject wavelength of radiation specific toconstituents in a sample;

FIG. 8 illustrates the design of a band pass filter in accordance withthe present invention (top portion) and the profile of the radiationpassing through the filter (bottom portion);

FIG. 9 illustrates the design of multi-modal band-pass or band-rejectfilters with corresponding intensity plots, in accordance with thepresent invention;

FIG. 10 illustrates the means for the intensity variation of a spectralfilter built in accordance with this invention;

FIGS. 11-14 illustrate alternative embodiments of a modulatingspectrometer in accordance with this invention; FIGS. 11A and 11B showembodiments in which the DMA is replaced with concave mirrors; FIG. 12illustrates an embodiment of a complete modulating spectrometer in whichthe DMA element is replaced by the concave mirrors of FIG. 11. FIG. 13illustrates a modulating lens spectrometer using lenses instead of DMA,and a “barber pole” arrangement of mirrors to implement variablemodulation. FIG. 14. illustrates a “barber pole” modulator arrangement;

FIGS. 15 and 16 illustrate an embodiment of this invention in which oneor more light sources provide several modulated spectral bands using afiber optic bundle;

FIG. 17 illustrates in diagram form an apparatus using controllableradiation source;

FIGS. 18A and 18B illustrate in a diagram form an optical synapseprocessing unit (OSPU) used as a processing element in accordance withthe present invention;

FIG. 19 illustrates in a diagram form the design of a spectrograph usingOSPU;

FIG. 20 illustrates in a diagram form an embodiment of a tunable lightsource;

FIG. 21 illustrates in a diagram form an embodiment of the spectralimaging device, which is built using two OSPUs;

FIGS. 22 and 23 illustrate different devices built using OSPUs;

FIGS. 24-26 are flow charts of various scans used in accordance with thepresent invention. Specifically, FIG. 24 is a flow chart of araster-scan used in one embodiment of the present invention; FIG. 25 isa flowchart of a Walsh-Hadamard scan used in accordance with anotherembodiment of the invention. FIG. 26 is a flowchart of a multi-scalescan, used in a different embodiment; FIG. 26A illustrates a multi-scaletracking algorithm in a preferred embodiment of the present invention;

FIG. 27 is a block diagram of a spectrometer with two detectors;

FIG. 28 illustrates a Walsh packet library of patterns for N=8.

FIG. 29 is a generalized block diagram of hyperspectral processing inaccordance with the invention;

FIG. 30 illustrates the difference in two spectral components (red andgreen) of a data cube produced by imaging the same object in differentspectral bands;

FIGS. 31A-31E illustrate different embodiments of an imagingspectrograph used in accordance with this invention in de-dispersivemode;

FIG. 32 shows an axial and a cross-sectional views of a fiber opticassembly;

FIG. 33 shows a physical arrangement of the fiber optic cable, detectorand the slit;

FIG. 34 illustrates a fiber optic surface contact probe head abuttingtissue to be examined;

FIGS. 35A and 35B illustrate a fiber optic e-Probe for pierced ears thatcan be used for medical monitoring applications in accordance with thepresent invention;

FIGS. 36A, 36B and 36C illustrate different configurations of ahyperspectral adaptive wavelength advanced illuminating imagingspectrograph (HAWAIIS) in accordance with this invention;

FIG. 37 illustrates a DMA search by splitting the scene;

FIG. 38 illustrates wheat spectra data (training) and wavelet spectrumin an example of determining protein content in wheat;

FIG. 39 illustrates the top 10 wavelet packets in local regression basisselected using 50 training samples in the example of FIG. 38;

FIG. 40 is a scatter plot of protein content (test data) vs. correlationwith top wavelet packet;

FIG. 41 illustrates PLS regression of protein content of test data;

FIG. 42 illustrates the advantage of DNA-based Hadamard Spectroscopyused in accordance with the present invention over the regular rasterscan;

FIGS. 43-47(A-D) illustrate hyperspectrum processing in accordance withthe present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

In one aspect, the present invention concerns the analysis of radiationpassing through or reflected from a sample of a material of interest.Since signal processing in this aspect of the invention is performedafter the sample has been irradiated, in the disclosure in Section Ibelow it is referred to as post-sample processing. Section II deals withthe aspect of the invention in which radiation has already beenprocessed prior to its interaction with the sample (e.g. based on apriori knowledge), and is accordingly referred to as pre-sampleprocessing. Various processing techniques applicable in both pre-sampleand post-sample processing are considered in Section III. Finally,Section IV illustrates the use of the proposed techniques and approachesin the description of various practical applications.

I.POST-SAMPLE PROCESSING

A. The Basic System

Turning now to the drawing figures and particularly FIGS. 1A and 1B, aspectrometer assembly 10 constructed in accordance with one embodimentof the invention is illustrated. With reference to FIG. 1A the devicebroadly includes a source 12 of electromagnetic radiation, a mirror andslit assembly 14, a wavelength dispersing device 16, a spatial lightmodulator 18, a detector 20, and an analyzing device 22.

In particular, the electromagnetic radiation source 12 is operable toproject rays of radiation onto or through a sample 24 that is to beanalyzed, such as a sample of body tissue or blood. The radiation sourcemay be any device that generates electromagnetic radiation in a knownwavelength spectrum such as a globar, hot wire, or light bulb thatproduces radiation in the infrared spectrum. To increase the amount ofrays that are directed to the sample, a parabolic reflector 26 may beinterposed between the source 12 and the sample 24. In a specificembodiment, the source of electromagnetic radiation is selected as toyield a continuous band of spectral energies, and is referred to as thesource radiation. It should be apparent that the energies of theradiation source are selected to cover the spectral region of interestfor the particular application.

The mirror and slit assembly 14 is positioned to receive the radiationrays from the source 12 after they have passed through the sample 24 andis operable to focus the radiation onto and through an entrance slit 30.The collection mirror 28 focuses the radiation rays through slit 30 andilluminates the wavelength dispersing device 16. As shown in diagramform in FIG. 1B, in different embodiments of the invention radiationrays from the slit may also be collected through a lens 15, beforeilluminating a wavelength dispersion device 16.

The wavelength dispersing device 16 receives the beams of radiation fromthe mirror and slit assembly 14 and disperses the radiation into aseries of lines of radiation each corresponding to a particularwavelength of the radiation spectrum. The preferred wavelengthdispersing device is a concave diffraction grating; however, otherwavelength dispersing devices, such as a prism, may be utilized. In aspecific embodiment, the wavelengths from the dispersing device 16 arein the near infrared portion of the spectrum and may cover, for example,the range of 1650-1850 nanometers (nm). It should be emphasized,however, that in general this device is not limited to just this or toany spectral region. It is intended that the dispersion device ingeneral is capable of operating in other ranges of electromagneticradiation, including the ultraviolet, visible, infrared, and microwavespectrum portions, as well as acoustic, electric, magnetic, and othersignals, where applicable.

The spatial light modulator (SLM) 18 receives radiation from thewavelength dispersing device 16, individually modulates each spectralline, and reflects the modulated lines of radiation onto the detector20. As illustrated in FIG. 2, the SLM is implemented in a firstpreferred embodiment as a micro-mirror array that includes asemi-conductor chip or piezo-electric device 32 having an array of smallreflecting surfaces 34 thereon that act as mirrors. One suchmicro-mirror array is manufactured by Texas Instruments and is describedin more detail in U.S. Pat. No. 5,061,049, hereby incorporated into thepresent application by reference. Those skilled in the art willappreciate that other spatial light modulators, such as a magneto-opticmodulator or a liquid crystal device may be used instead of themicro-mirror array. Various embodiments of such devices are discussed inmore detail below.

The semi-conductor 32 of the micro-mirror array 18 is operable toindividually tilt each mirror along its diagonal between a firstposition depicted by the letter A and a second position depicted by theletter B in FIG. 3. In preferred forms, the semi-conductor tilts eachmirror 10 degrees in each direction from the horizontal. The tilting ofthe mirrors 34 is preferably controlled by the analyzing device 22,which may communicate with the micro-mirror array 18 through aninterface 37.

The micro-mirror array 18 is positioned so that the wavelengthdispersing device 16 reflects each of the lines of radiation upon aseparate column or row of the array. Each column or row of mirrors isthen tilted or wobbled at a specific and separate modulation frequency.For example, the first row of mirrors may be wobbled at a modulationfrequency of 100 Hz, the second row at 200 Hz, the third row at 300 Hz,etc.

In a specific embodiment, the mirrors are calibrated and positioned sothat they reflect all of the modulated lines of radiation onto adetector 20. Thus, even though each column or row of mirrors modulatesits corresponding line of radiation at a different modulation frequency,all of the lines of radiation are focused onto a single detector.

The detector 20, which may be any conventional radiation transducer orsimilar device, is oriented to receive the combined modulated lines ofradiation from the micro-mirror array 18. The detector is operable forconverting the radiation signals into a digital output signal that isrepresentative of the combined radiation lines that are reflected fromthe micro-mirror array. A reflector 36 may be interposed between themicro-mirror array 18 and the detector 20 to receive the combinedmodulated lines of radiation from the array and to focus the reflectedlines onto the detector.

The analyzing device 22 is operably coupled with the detector 20 and isoperable to receive and analyze the digital output signal from thedetector. The analyzing device uses digital processing techniques todemodulate the signal into separate signals each representative of aseparate line of radiation reflected from the micro-mirror array. Forexample, the analyzing device may use discrete Fourier transformprocessing to demodulate the signal to determine, in real time, theintensity of each line of radiation reflected onto the detector. Thus,even though all of the lines of radiation from the micro-mirror arrayare focused onto a single detector, the analyzing device can separatelyanalyze the characteristics of each line of radiation for use inanalyzing the composition of the sample.

In accordance with one embodiment of this invention, the analyzingdevice is preferably a computer that includes spectral analysissoftware. FIG. 4 illustrates an output signal generated by the analyzingdevice in accordance with one embodiment. The output signal illustratedin FIG. 4 is a plot of the absorption characteristics of fivewavelengths of radiation from a radiation source that has passed througha sample.

In one embodiment of the system of this invention illustrated in FIG.6A, it is used for digital imaging purposes. In particular, when used asan imaging device, an image of a sample 38 is focused onto amicro-mirror array 40 and each micro-mirror in the array is modulated ata different modulation rate. The micro-mirror array geometry is suchthat some or all of the reflected radiation impinges upon a singledetector element 42 and is subsequently demodulated to reconstruct theoriginal image improving the signal-to-noise ratio of the imager.Specifically, an analyzing device 44 digitally processes the combinedsignal to analyze the magnitude of each individual pixel. FIG. 6Billustrates spatio-spectral distribution of the DMA, where individualelements can be modulated. FIG. 5 is a plot of a three dimensional imageshowing the magnitude of each individual pixel.

FIG. 7 illustrates the output of a digital micro-mirror array (DMA)filter spectrometer used as a variable band pass filter spectrometer,variable band reject filter spectrometer, variable multiple band passfilter spectrometer or variable multiple band reject filterspectrometer. In this embodiment, the combined measurement of theelectromagnetic energy absorbed by sample constituents A and C is ofinterest. The shaded regions in FIG. 7 illustrate the different regionsof the electromagnetic spectrum that will be allowed to pass to thedetector by the DMA filter spectrometer. The wavelengths ofelectromagnetic radiation selected to pass to the detector correspond tothe absorption band for compound A and absorption band for compound C ina sample consisting of compounds A, B, and C. The spectral regioncorresponding to the absorption band of compound B and all otherwavelengths of electromagnetic radiation are rejected. Those skilled inthe art will appreciate that the DMA filter spectrometer is not limitedto the above example and can be used to pass or reject any combinationof spectral resolution elements available to the DMA. Various examplesand modifications are considered in detail below.

As a DMA filter imager the spatial resolution elements (pixels) of animage can be selectively passed or rejected (filtered) according to therequirements of the image measurement. The advantages of both the DMAfilter spectrometer and DMA filter imager are:

(1) All spectral resolution elements or spatial resolution elementscorresponding to the compounds of interest in a particular sample can bedirected simultaneously to the detector for measurement. This has theeffect of increasing the signal-to-noise ratio of the measurement.

(2) The amount of data requiring processing is reduced. This reducesstorage requirements and processing times.

B. Modulated Spectral Filter Design

(i) Design Basics

The preceding section described the components of the basic system usedin accordance with the present invention, and their operation. The focusof this section is on the design of specific modulated spectral filtersusing the spatial light modulator (SLM) 18, which in a preferredembodiment is implemented using a digital micro-mirror array (DMA).

As noted above, using a DMA one can provide one or more spectral bandpass or band-reject filter(s) with a chosen relative intensity. Inparticular, in accordance with the present invention the radiationwavelengths that are reflected in the direction of the detector areselected by specific columns of micro-mirrors of the DMA, as illustratedin FIG. 8. The relative intensity of the above spectral band iscontrolled by the selection of specific area of micro-mirrors on theDMA, represented by the dark area designated “A” in FIG. 8. Thus, thedark area shown in FIG. 8 is the mirrors that direct specific wavelengthradiation, i.e., spectral band, to the detector. Clearly, the “on”mirrors in the dark area create a band-pass filter, the characteristicsof which are determined by the position of the “on” area in the DMA. Thebottom portion of the figure illustrates the profile of the radiationreaching the detector.

FIG. 8 also demonstrates the selection of specific rows and columns ofmirrors in the DMA used to create one spectral band filter with a singlespectral mode. It should be apparent, however, that using the sametechnique of blocking areas in the DMA one can obtain a plurality ofdifferent specific spectral band filters, which can have multi-modalcharacteristics. The design of such filters is illustrated in FIG. 9.

As shown in FIG. 9, a multitude of different specific filters can bedesigned on one DMA using simple stacking. FIG. 9 illustrates thecreation of several filters by selective reflection from specificmicro-mirrors. In particular, the left side of the figure illustratesthe creation of three different filters, designated 1, 2, and 3. This isaccomplished by the selection of specific mirrors on the DMA, asdescribed above with reference to FIG. 8. The total collection ofspectral band filters is shown at the bottom-left of this figure. Thespectral band provided by each filter is shown on the right-hand side ofthe figure. The bottom right portion illustrates the radiation passingthrough the combination of filters 1, 2 and 3.

The above discussion describes how the relative intensity of eachspectral band can be a function of the DMA area used in the reflection.The following table illustrates the linear relationship between areas ofthe DMA occupied by individual filters, and the resulting filter.Clearly, if the entire DMA array is in the “on” position, there will beno filtering and in principle the input radiation passes through with noattenuation. FIG. 9, left side FIG. 9, right side Reflected radiationfrom micro-mirrors Filter created area A 1 area B 2 area C 3 areas a +b + c 1 + 2 + 3

FIG. 10 illustrates the means for the intensity variation of a spectralfilter built in accordance with this invention, and is summarized in thetable below. Example A Example B Reflection from a DMA The intensityrecorded at the See FIGS. 8 and 9. detector for example A for theReflection areas 1 , 2, combination filter 1, 2, and 3, and 3 createspectral filter Intensity, I, I₁ = I₂ = I₃ 1, 2 and 3 respectively. area1 = area 2 = area 3 Example C Example D The reflection of area 2 of Theintensity recorded at the the DMA is increased. detector for filters 1,2, and area 1 = area 3 < area 2 3 is I₁ ˜ I₃ < I₂ Example E Example FThe reflection of area 2 of The intensity recorded at the the DMA isdecreased detector for filter 1, 2, and area 1 = area 3 < area 2 3 is I₁= I₃ < I₂

(ii) Modulation

FIGS. 9 and 10 illustrate the ability to design spectral filters withdifferent characteristics using a DMA. The important point to keep inmind is that different spectral components of the radiation from thesample have been separated in space and can be filtered individually. Itis important to retain the ability to process individual spectralcomponents separately. To this end, in accordance with the presentinvention, spectral components are modulated.

The basic idea is to simply modulate the output from different filtersdifferently, so one can identify and process them separately. In apreferred embodiment, different modulation is implemented by means ofdifferent modulation rates. Thus, with reference to FIG. 9, the outputof filter 1 is modulated at rate M₁; output of filter 2 is modulated atrate M₂, and filter 3 is modulated using rate M₃, where M₁ ≢M₂ ≢M₃. Indifferent embodiments, modulation may be achieved by assigning adifferent modulation encodement to each filter, with which it ismodulated over time.

As a result, a system built in accordance with the present invention iscapable of providing: a) Spectral bandwidth by selection of specificcolumns of micro-mirrors in an array; b) Spectral intensity by selectionof rows of the array; and c) Spectral band identification by modulation.All of the above features are important in practical applications, asdiscussed in Section IV below.

C. Alternative Embodiments

(i) Modulating Spectrometers without a DMD.

FIGS. 11-14 illustrate alternative embodiments of a modulatingspectrometer in accordance with this invention, where the DMA isreplaced with different components. In particular, FIGS. 11A and B showan embodiment in which the DMA is replaced with fixed elements, in thiscase concave mirrors. The idea is to use fixed spectral grating, whichmasks out spectrum block components that are not needed and passes thosewhich are.

The idea here is that the broadly illuminated dispersive elementdistributes spectral resolution elements in one dimension so that in theorthogonal dimension one can collect light of the same wavelengths. Withreference to FIG. 6A one can see that at a particular defined plane,herein called the focal plane, one has a wavelength axis(x or columns)and a spatial axis (y or rows). If one were to increase the number ofspatial resolution elements (y) that are allowed to pass energy throughthe system and out of the exit aperture for any given wavelength (x), orspectral resolution element (x), this would have the effect ofincreasing the intensity of the particular spectral resolution elements'intensity at the detector.

If the array of spatio/spectral resolution elements at the focal planeas shown in FIG. 6A is replaced with fixed elements, such as the concavemirrors in FIG. 11B, one can have a different device configured toperform a particular signal processing task—in this case pass thepredetermined spectrum components at the desired intensity levels. FIG.11A shows the spatio/spectral resolution elements at the focal plane tohe used. The fixed optical elements are placed to interact withpredetermined spatio/spectral resolution elements provided by thegrating and entrance aperture geometry and to direct the specificassortment of spatio/spectral elements to specific spatial locations formodulation encoding (possibly using the barber pole arrangement, shownnext).

FIG. 12 illustrates an embodiment of a complete modulating spectrometerin which the DMA element is replaced by the concave mirrors of FIG. 11.FIG. 13 illustrates a modulating lens spectrometer using lenses insteadof DMA, and a “barber pole” arrangement of mirrors to implement variablemodulation. The “barber pole” modulation arrangement is illustrated inFIG. 14.

With reference to FIG. 14, modulation is accomplished by rotating this“barber pole” that has different number of mirrors mounted forreflecting light from the spatially separated spectral wavelengths.Thus, irradiating each vertical section will give the reflector its owndistinguishable frequency. In accordance with this embodiment, lightfrom the pole is collected and simultaneously sent to the detector.Thus, radiation from concave mirror 1 impinges upon the four-mirrormodulator; concave mirror 2 radiation is modulated by the five-mirrormodulator, and concave mirror 3 directs radiation to the six-mirrormodulator. In the illustrated embodiment, the modulator rate is four,five, or six times per revolution of the “barber pole.”

The operation of the device is clarified with reference to FIG. 12,tracing the radiation from the concave mirrors 12 to the detector of thesystem. In particular, concave mirror 1 reflects a selected spectralband with chosen intensity. This radiated wave impinges upon amodulator, implemented in this embodiment as a rotation barber pole. Themodulating rates created by the barber pole in the exemplary embodimentshown in the figure are as shown in the table below. Number of mirrorsModulation Per 360° rotation Per 360° of barber pole Area A 4 4/360°Area B 5 5/360° Area C 6 6/360°

Accordingly, this arrangement yields a modulation rate of 4/360° for theradiation from Area A, FIG. 12.

By a analogy, the mirrors of Areas B and C are modulated at the rate of5/360° and 6/360°, respectively. As illustrated, all radiation frommirrors A, B, and C is simultaneously directed to the detector. Thisradiation is collected by either a simple mirror lens or a toroidalmirror, which focuses the radiation onto a single detector. The signalfrom the detector now goes to electronic processing and mathematicalanalyses for spectroscopic results.

(iii) Modulating Light Sources Spectrometer.

In the discussion of modulating spectrometers, a single light source ofelectromagnetic radiation was described. There exist yet anotherpossibility for a unique optical design—a modulating multi-light sourcespectrometer. FIGS. 15 and 16 illustrate an embodiment of this inventionin which a light source 12 provides several modulated spectral bands,e.g., light emitting diodes (LED), or lasers (shown here in threedifferent light sources). The radiation from these light sourcesimpinges upon the sample 24. One possible illumination design is one inwhich light from a source, e.g. LED, passes through a multitude offilters, impinging upon the sample 24. The radiation from the sample istransmitted to a detector 20, illustrated as a black fiber. The signalfrom the detector is electronically processed to a quantitative andqualitative signal describing the sample chemical composition.

In this embodiment, a plurality of light sources is used at differedmodulating rates. FIGS. 15 and 16 illustrate the combination of severallight sources in the spectrometer. The choice of several differentspectral bands of electromagnetic radiation can be either light emittingdiodes, LED, lasers, black body radiation and/or microwaves. Essentiallythe following modulation scheme can be used to identify the differentlight sources, in this example LED's of different spectral bandwavelength. No. of Spectral band Modulation Source Wavelength, nm Rate 11500-1700 m₁ 2 1600-1800 m₂ 3 1700-1900 m₃ . . . . . . . . .Note:m₁ ≠ m₂ ≠ m₃ ≠ . . .

It should be noted that either the radiation will be scattered ortransmitted by the sample 24. This scattered or transmitted radiationfrom the sample is collected by an optical fiber. This radiation fromthe sample is conducted to the detector. The signal from the detector iselectronically processed to yield quantitative and qualitativeinformation about the sample.

In a particular embodiment the radiation path consists of opticalfibers. However, in accordance with alternate embodiments, mirrors andlenses could also constitute the optical path for a similar modulatingmulti-light source spectrometer.

(iv) Modulating Multi-source Hyperspectral Imaging Spectrometer

The spectrometer described in the preceding section records spectralinformation about one unique area on a single detector. In a similarmanner, the spectral characteristic of a multitude of areas in a samplecan be recorded with a multitude of detectors in accordance withdifferent embodiments of the invention. Such a multitude of detectorsexists in an array detector. Array detectors are known in the art andinclude, for example Charge coupled devices (CCD), in the ultraviolet,and visible portions of the spectrum; InSb—array in near infrared;InGaAs—array in near infrared; Hg—Cd—Te—array in mid-infrared and otherarray detectors.

Array detectors can operate in the focal plane of the optics. Here eachdetector of the array detects and records the signal from a specificarea, x_(i)y_(i). Practical Example B in Section IV on the gray-levelcamera provides a further illustration. Different aspects of theembodiments discussed in sections (iii) and (iv) are considered in moredetail in the following sections. As is understood by one skilled in theart, standard optical duality implies that each of the precedingconfigurations can be operated in reverse, exchanging the position ofthe source and the detector.

II. PRE-SAMPLE PROCESSING

The preceding section described an aspect of the invention referred toas post-sample processing, i.e., signal processing performed after asample had been irradiated. In accordance with another important aspectof this invention, significant benefits can result from irradiating asample with pre-processed radiation, in what is referred to aspre-sample processing. Most important in this context is the use, inaccordance with this invention, of one or more light sources, capable ofproviding modulated temporal and/or spatial patterns of input radiation.These sources are referred to next as controllable source(s) ofradiation, which in general are capable of generating arbitrarycombinations of spectral radiation components within a predeterminedspectrum range.

Several types of prior art devices are known that are capable ofproviding controllable radiation. Earlier prior art devices primarilyrelied upon various “masking” techniques, such as electronicallyalterable masks interposed in the optical pathway between a light sourceand a detector. More recent prior art devices use a combination of twoor more light-emitting diodes (LEDs) as radiation sources. Examples areprovided in U.S. Pat. Nos. 5,257,086 and 5,488,474, the content of whichis hereby incorporated by reference for all purposes. As discussed inthe above patents, an array of LEDs or light-emitting lasers isconfigured for activation using a particular encoding pattern, and canbe used as a controllable light source. A disadvantage of this system isthat it relies on an array of different LED elements, each operating ina different, relatively narrow spectrum band. In addition, there aretechnological problems associated with having an array of discreteradiation elements with different characteristics.

These and other problems associated with the prior art are addressed inaccordance with the present invention using a device that in a specificembodiment can be thought of as the reverse of the setup illustrated inFIG. 1A. In particular, one or more broadband radiation sourcesilluminate the digital micro-mirror array (DMA) 18 and the modulationsof the micro-mirrors in the DMA encode the source radiation prior toimpinging upon the sample. The reflected radiation is then collectedfrom the sample and directed onto a detector for further processing.

FIG. 17 illustrates a schematic representation of an apparatus inaccordance with the present invention using a controllable radiationsource. Generally, the system includes a broadband radiation source 12,DMA 18, wavelength dispersion device 16, slit assembly 30, detector 20and control assembly 22.

In particular, control assembly 22 may include a conventional personalcomputer 104, interface 106, pattern generator 108, DMA driver 110, andanalog to digital (A/D) converter 114. Interface 106 operates as aprotocol converter enabling communications between the computer 22 anddevices 108-114.

Pattern generator 108 may include an EPROM memory device (not shown)which stores the various encoding patterns for array 18, such as theHadamard encoding pattern discussed below. In response to controlsignals from computer 22, generator 108 delivers signals representativeof successive patterns to driver 110. More particularly, generator 108produces output signals to driver 110 indicating the activation patternof the mirrors in the DMA 18. A/D converter 114 is conventional innature and receives the voltage signals from detector 20, amplifiesthese signals as analog input to the converter in order to produce adigital output representative of the voltage signals.

Radiation source 12, grating 16, DMA 18 slit assembly 30 and detector 20cooperatively define an optical pathway. Radiation from source 12 ispassed through a wavelength dispersion device, which separates in spacedifferent spectrum bands. The desired radiation spectrum can them beshaped by DMA 18 using the filter arrangement outlined in SectionI(B)(i). In accordance with a preferred embodiment, radiation falling ona particular micro-mirror element can also be encoded with a modulationpattern applied to it. In a specific mode of operating the device, DMA18 is activated to reflect radiation in a successive set of encodingpatterns, such as Hadamard, Fourier, wavelet or others. The resultantset of spectral components is detected by detector 20, which providescorresponding output signals. Computer 22 then processes these signals.

Computer 22 initiates an analysis by prompting pattern generator 108 toactivate the successive encoding patterns. With each pattern, a set ofwavelength components are resolved by grating 16 and after reflectionfrom the DMA 18 is directed onto detector 20. Along with the activationof encoding patterns, computer 22 also takes readings from A/D converter114, by sampling data. These readings enable computer 22 to solve aconventional inverse transform, and thereby eliminate background noisefrom the readings for analysis.

In summary, the active light source in accordance with the presentinvention consists of one or more light sources, from which variousspectral bands are selected for transmission, while being modulated witha temporal and/or spatial patterns. The resulting radiation is thendirected at a region (or material) of interest to achieve a variety ofdesired tasks. A brief listing of these tasks include: (a) Very precisespectral coloring of a scene, for purposes of enhancement of display andphotography; (b) Precise illumination spectrum to correspond to specificabsorption lines of a compound that needs to be detected, (see FIGS.38-42 on protein in wheat as an illustration) or for which it isdesirable to have energy absorption and heating, without affectingneighboring compounds (This is the principle of the microwave oven forwhich the radiation is tuned to be absorbed by water molecules allowingfor heating of moist food only); (c) The procedure in (b) could be usedto imprint a specific spectral tag on ink or paint, for watermarking,tracking and forgery prevention, acting as a spectral bar codeencryption; (d) The process of light curing to achieve selected chemicalreactions is enabled by the tunable light source.

Various other applications are considered in further detail in SectionIV. Duality allows one to reverse or “turn inside out” any of thepost-sample processing configurations described previously, to yield apre-sample processing configuration. Essentially, in the former case onetakes post sample light, separates wavelengths, encodes or modulateseach, and detects the result. The dualized version for the latter caseis to take source light, separates wavelengths, encode or modulate each,interact with a sample, and detect the result.

III. OPTICAL ENCODING, DECODING AND SIGNAL PROCESSING

The preceding two sections disclosed various embodiments of systems forperforming post- and pre-sample processing. In a specific embodiment,the central component of the system is a digital micro-mirror array(DMA), in which individual elements (micro-mirrors) can be controlledseparately to either pass along or reject certain radiation components.By the use of appropriately selected modulation patterns, the DMA arraycan perform various signal processing tasks. In a accordance with apreferred embodiment of this invention, the functionality of the DMAsdiscussed above can be generalized using the concept of Spatial LightModulators (SLMs), devices that broadly perform spatio-spectral encodingof individual radiation components, and of optical synapse processingunits (OSPUs), basic processing blocks. This generalization isconsidered in subsection III.A, followed by discussions of Hadamardprocessing, spatio-spectral tagging, data compression, featureextraction and other signal processing tasks.

A. Basic Building Blocks

(i) Spatial Light Modulators (SLMs)

In accordance with the present invention, one-dimensional (1D),two-dimensional (2D) or three-dimensional (3D) devices capable of actingas a light valve or array of light valves are referred to as spatiallight modulators (SLMs). More broadly, an SLM in accordance with thisinvention is any device capable of controlling the magnitude, power,intensity or phase of radiation or which is otherwise capable ofchanging the direction of propagation of such radiation. This radiationmay either have passed through, or be reflected or refracted from amaterial sample of interest. In a preferred embodiment, an SLM is anarray of elements, each one capable of controlling radiation impingingupon it. Note that in accordance with this definition an SLM placed inappropriate position along the radiation path can control either spatialor spectral components of the impinging radiation, or both. Furthermore,“light” is used here in a broad sense to encompass any portion of theelectromagnetic spectrum and not just the visible spectrum. Examples ofSLM's in accordance with different embodiments of the invention includeliquid crystal devices, actuated micro-mirrors, actuated mirrormembranes, di-electric light modulators, switchable filters and opticalrouting devices, as used by the optical communication and computingenvironments and optical switches. In a specific embodiment, Sections IAand IB discussed the use of a DMA as an example of spatial lightmodulating element. U.S. Pat. No. 5,037,173 provides examples oftechnology that can be used to implement SLM in accordance with thisinvention, and is hereby incorporated by reference.

In a preferred embodiment, a 1D, 2D, or 3D SLM is configured to receiveany set of radiation components and functions to selectively pass thesecomponents to any number of receivers or image planes or collectionoptics, as the application may require, or to reject, reflect or absorbany input radiation component, so that either it is or is not receivedby one or more receivers, image planes or collection optics devices. Itshould be clear that while in the example discussed in Section I abovethe SLM is implemented as a DMA, virtually any array of switchedelements may be used in accordance with the present invention.

Generally, an SLM in accordance with the invention is capable ofreceiving any number of radiation components, which are then encoded,tagged, identified, modulated or otherwise changed in terms of directionand/or magnitude to provide a unique encodement, tag, identifier ormodulation sequence for each radiation component in the set of radiationcomponents, so that subsequent optical receiver(s) or measuringdevice(s) have the ability to uniquely identify each of the inputradiation components and its properties. In a relevant context, suchproperties include, but are not limited to, irradiance, wavelength, bandof frequencies, intensity, power, phase and/or polarization. In SectionsI and II above, tagging of individual radiation components isaccomplished using rate modulation. Thus, in Section I, differentspectral components of the input radiation that have been separated inspace using a wavelength dispersion device are then individually encodedby modulating the micro-mirrors of the DMA array at different rates. Theencoded radiation components are directed to a single detector, butnevertheless can be analyzed individually using Fourier analysis of thesignal from the detector. Other examples for the use of “tagging” arediscussed below.

(ii) The Optical Synapse Processing Unit (OSPU)

In accordance with this invention, various processing modalities can berealized with an array of digitally controlled switches (an opticalsynapse), which function to process and transmit signals betweendifferent components of the system. In the context of the abovedescription, the basic OSPU can be thought of as a data acquisition unitcapable of scanning an array of data, such as an image, in variousmodes, including raster, Hadamard, multiscale wavelets, and others, andtransmitting the scanned data for further processing. Thus, a synapse isa digitally controlled array of switches used to redirect image (orgenerally data) components or combinations of light streams, from onelocation to one or more other locations. In particular it can performHadamard processing, as defined below, on a plurality of radiationelements by combining subsets of the elements (i.e., binning) beforeconversion to digital data. A synapse can be used to modulate lightstreams by modulating temporally the switches to impose a temporal barcode (by varying in time the binning operation). This can be built in apreferred embodiment from a DMA, or any of a number of optical switchingor routing components, used for example in optical communicationsapplications.

An OSPU unit in accordance with the present invention is shown indiagram form in FIGS. 18A and 18B, as three-port device taking inputfrom a radiation source S, and distributing it along any of two otherpaths, designated C (short for camera) and D (for detector). Differentscanning modes of the OSPU are considered in more detail in SectionIII.B. below.

In the above disclosure and in one preferred embodiment of the inventionan OSPU is implemented using a DMA, where individual elements of thearray are controlled digitally to achieve a variety of processing taskswhile collecting data. In accordance with the present invention,information bearing radiation sources could be, for example, a stream ofphotons, a photonic wavefront, a sound wave signal, an electricalsignal, a signal propagating via an electric field or a magnetic field,a stream of particles, or a digital signal. Example of devices that canact as a synapse include spatial light modulators, such as LCDs, MEMSmirror arrays, or MEMS shutter arrays; optical switches; opticaladd-drop multiplexers; optical routers; and similar devices configuredto modulate, switch or route signals. Clearly, DMAs and other opticalrouting devices, as used by the optical communication industry can beused to this end. It should be apparent that liquid crystal displays(LCD), charge coupled devices (CCD), CMOS logic, arrays of microphones,acoustic transducers, or antenna elements for electromagnetic radiationand other elements with similar functionality that will be developed inthe future, can also be driven by similar methods.

Applicants' contribution in this regard is in the novel process ofperforming pre-transduction digital computing on analog data viaadaptive binning means. Such novelty can be performed in a large numberof ways. For example, one can implement adaptive current addition usinga parallel/serial switch and wire networks in CMOS circuits. Further, inthe acoustic processing domain, one or more microphones can be used incombination with an array of adjustable tilting sound reflectors (like aDMD for sound). In each case, one can “bin” data prior to transduction,in an adaptive way, and hence measure some desired computational resultthat would traditionally be obtained by gathering a “data cube” of data,and subsequently digitally processing the data. The shift of paradigm isclear: in the prior art traditionally analog signals are captured by asensor, digitized, stored in a computer as a “data cube”, and thenprocessed. Considerable storage space and computational requirements areextended to do this processing. In accordance with the presentinvention, data from one or more sensors is processed directly in theanalogue domain, the processed result is digitized and sent to acomputer, where the desired processing result may be available directly,or following reduced set of processing operations.

In accordance with the present invention, the digitally controlled arrayis used as a hybrid computer, which through the digital control of thearray elements performs (analog) computation of inner products or moregenerally of various correlations between data points reaching theelements of the array and prescribed patterns. The digital control at agiven point (i.e., element) of the array may be achieved through avariety of different mechanisms, such as applying voltage differencesbetween the row and column intersecting at the element; the modulationis achieved by addressing each row and column of the array by anappropriately modulated voltage pattern. For example, when using DMA,the mirrors are fluctuating between two tilted positions, and modulationis achieved through the mirror controls, as known in the art. Thespecifics of providing to the array element of signal(s) following apredetermined pattern will depend on the design implementation of thearray and are not considered in further detail. Broadly, the OSPU arrayis processing raw data to extract desired information.

In accordance with the present invention, various assemblies of OSPUalong with other components can be used to generalize the ideaspresented above and enable new processing modalities. For example, FIG.19 illustrates in block diagram form the design of a spectrograph usingOSPU. As shown, the basic design brings reflected or transmittedradiation from a line in the sample or source onto a dispersing device16, such as a grating or prism, onto the imaging fiber into the OSPU toencode and then forward to a detector 20.

FIG. 20 illustrates in a diagram form an embodiment of a tunable lightsource, which operates as the spectrograph in FIG. 19, but uses abroadband source. In this case, the switching elements of the OSPUarray, for example the mirrors in a DMA, are set to provide a specifiedenergy in each row of the mirror, which is sent to one of the outgoingimaging fiber bundles. This device can also function as a spectrographthrough the other end, i.e., fiber bundle providing illumination, aswell as spectroscopy.

FIG. 21 illustrates in a diagram form an embodiment of the spectralimaging device discussed in Section I above, which is built with twoOSPUs. Different configurations of generalized processing devices areillustrated in FIG. 22, in which each side is imaging in a differentspectral band, and FIG. 23, which illustrates the main components of asystem for processing input radiation using an OSPU.

B. Scanning an Area of Interest

In accordance with the present invention, different scanning modes canbe used in different applications, as illustrated in FIG. 24, FIG. 25and FIG. 26. These algorithms are of use, for example, when one is usingan OSPU in conjunction with a single sensor, and the OSPU is binningenergy into that sensor, the binning being determined by the patternthat is put onto the SLM of the OSPU.

In particular, FIG. 24 is a flow chart of a raster-scan using in oneembodiment of the present invention. This algorithm scans a rectangle,the “Region Of Interest (ROI),” using ordinary raster scanning. It isintended for use in configurations in this disclosure that involve aspatial light modulator (SLM). It is written for the 2D case, but theobvious modifications will extend the algorithm to other dimensions, orrestrict to 1D.

FIG. 25 is a flowchart of a Walsh-Hadamard scan used in accordance withanother embodiment of the invention. This algorithm scans a rectangle,the “Region Of Interest (ROI)”, using Walsh-Hadamard multiplexing.Walsh( dx, m, i, dy, n, j) is the Walsh-Hadamard pattern with origin(dx, dy), of width 2^(m) and height 2^(n), horizontal Walsh index i, andvertical Walsh index j.

FIG. 26 is a flowchart of a multi-scale scan. This algorithm scans arectangle, the “Region Of Interest (ROI)”, using a multi-scale search.It is intended for use in a setting as in the description of the rasterscanning algorithm. The algorithm also presumes that a procedure existsfor assigning a numerical measure to the pattern that is currently on iscalled an “interest factor.”

FIG. 26A illustrates a multi-scale tracking algorithm in a preferredembodiment of the present invention. The algorithm scans the region ofinterest, (using multi-scan search), to find an object of interest andthen tracks the object's movement across the scene. It is intended foruse in a setting where multi-scale search can be used, and where the“interest factor” is such that a trackable object can be found. Examplesof interest factors used in accordance with a preferred embodiment (whenpattern L_(i) is put onto the SLM, the sensor reads C_(i) and we aredefining the “interest factor” F_(i)). In the preceding scan algorithmsa single sensor is assumed. Thus

1. F(L_(i))=C_(i)

2. F(L_(i))=C_(i)/area(L_(i))

3. F(L_(i))=C_(i)/C_(k), where L_(k) is the rectangle that containsL_(i), and that has N times the area of L_(i), (for example, N=4), andwhich has already been scanned by the algorithm (there will always beexactly one such).

A modification of the algorithm is possible, where instead of putting upthe pattern L_(i), one can put up a set of a few highly oscillatoryWalsh patterns fully supported on exactly L_(i), and take the mean valueof the sensor reading as F_(i). This estimates the total variationwithin L_(i) and will yield an algorithm that finds the edges within ascene. In different examples the sensor is a spectrometer.F(L_(i))=distance between the spectrum read by the sensor, and thespectrum of a compound of interest. (distance could be, e.g., Euclideandistance of some other standard distance). This will cause the algorithmto zoom in on a substance of interest.

In another embodiment, F(L_(i))=distance between the spectrum read bythe sensor, and the spectrum already read for L_(k), where L_(k) is therectangle that contains L_(i), and that has N (N=4) times the area ofL_(i), and which has already bee scanned by the algorithm (there willalways be exactly one such). This will cause the algorithm to zoom in onedges between distinct substances.

In yet another embodiment, F(L_(i))=distance between the spectrum readby the sensor, and the spectrum already read for L₀. This will cause thealgorithm to zoom in on substances that are anomalous compared to thebackground.

In derived embodiments, F(L_(i)) can depend on a priori data fromspectral or spatio-spectral libraries.

By defining the interest factor appropriately, one can thus cover arange of different applications. In a preferred embodiment, the interestfactor definitions can be pre-stored so a user can analyze a set of datausing different interest factors.

It is also clear that, in the case of Walsh functions, because of themulti-scale nature of the Walsh patterns, one can combine raster andWalsh-Hadamard scanning (raster scanning at large scales, and usingWalsh-Hadamard to get extra signal to noise ratio at fine scales, whereit is needed most). This allows one to operate within the linear rangeof the detector.

Also, one can used the combined raster/Walsh idea in variations of theMulti-scale search and tracking algorithms. For this, whenever one isstudying the values of a sensor associated with the sub-rectangles of abigger rectangle, one could use the Walsh patterns at the relevantscale, instead of scanning the pixels at that scale. This will providefor an improvement in SNR. One could again do this only at finer scales,to stay in the detectors linearity range.

C. Hadamard and Generalized Hyperspectral Processing

Several signal processing tasks, such as filtering, signal enhancement,feature extraction, data compression and others can be implementedefficiently by using the basic ideas underlying the present invention.The concept is first illustrated in the context of one-dimensionalarrays for Hadamard spectroscopy and is then extended to hyperspectralimaging and various active illumination modes. The interested reader isdirected to the book “Hadamard Transform Optics” by Martin Harwit, etal., published by Academic Press in 1979, which provides an excellentoverview of the applied mathematical theory and the degree to whichcommon optical components can be used in Hadamard spectroscopy andimaging applications.

Hadamard processing refers generally to analysis tools in which a signalis processed by correlating it with strings of 0 and 1 (or +/−1). Suchprocessing does not require the signal to be converted from analogue todigital, but permits direct processing on the analog data by means of anarray of switches (synapse). In a preferred embodiment of the invention,an array of switches, such as a DMA, is used to provide spatio-spectraltags to different radiation components. In alternative embodiments itcan also be used to impinge spatio/spectral signatures, which directlycorrelate to desired features.

A simple way to explain Hadamard spectroscopy is to consider the exampleof the weighing schemes for a chemical scale. Assume that we need toweigh eight objects, x₁, x₂ . . . x₈, on a scale. One could weigh eachobject separately in a process analogous to performing a raster scan, orbalance two groups of four objects. Selecting the second approach,assuming that the first four objects are in one group, and the secondfour in a second group, balancing the two groups can be representedmathematically using the expression:m=x+x ₂ +x ₃ +x ₄−(x ₅ +x ₆ +x ₇ +x ₈)=(x, w),

where x is a vector, the components of which correspond to the orderedobjects xi,=(1,1,1,1,−1,−1,−1,−1) and (x, w) designates the innerproduct of the two vectors. Various other combinations of object groupscan be obtained and mathematically expressed as the inner product of thevector x and a vector of weights w, which has four +1 and four −1elements.

For example, w=(1, −1, 1, 1, −1, −1, 1, −1) indicates that x₁,x₃,x₄,x₇are on the left scale while x₂ x₅ x₆ x₈ are on the right. The innerproduct, or weight M =(x, w) is given by the expression:m=(x,w)=x ₁ −x ₂ +x ₃ +x ₄ −x ₅ −x ₆ +x ₇ −x ₈

It is well known that if one picks eight mutually orthogonal vectorsw_(i) which correspond, for example, to the eight Walsh patterns, onecan recover the weight x_(i) of each object via the orthogonal expansionmethodx=[(x, w ₁)w ₁+(x, w ₂)w ₂+. . . +(x, w ₈)w ₈],

or in matrix notation[W]x=m; x=[W] ⁻¹ m

where [W] is the matrix of orthogonal vectors, m is the vector ofmeasurements, and [W]⁻¹ is the inverse of matrix [W].

It is well known that the advantage of using the method is itshigher-accuracy, more precisely if the error for weighing measurement isε, the expected error for the result calculated from the combinedmeasurements is reduced by the square root of the number of samples.This result was proved by Hotteling to provide the best reductionpossible for a given number of measurements.

In accordance with the present invention, this signal processingtechnique finds simple and effective practical application inspectroscopy, if we consider a spectrometer with two detectors(replacing the two arms of the scales). With reference to FIG. 27, thediffraction grating sends different spectral lines into an eight mirrorarray, which redistributes the energy to the 2 detectors in accordancewith a given pattern of +1/−1 weights, i.e.,w_(i)=(1,−1,1,1,−1,−1,1,−1). Following the above analogy, the differencebetween the output values of the detectors corresponds to the innerproduct m=(x,w_(i)). If one is to redistribute the input spectrum energyto the 2 spectrometers using eight orthogonal vectors of weights,(following the pattern by alternating the mirror patterns to get eightorthogonal configurations), an accurate measurement of the sourcespectrum can be obtained. This processing method has certain advantagesto the raster scan in which the detector measures one band at a time.

Clearly, for practical applications a precision requiring hundreds ofbands may be required to obtain accurate chemical discrimination.However, it should be apparent that if one knows in advance which bandsare needed to discriminate two compounds, the turning of the mirrors toonly detect these bands could provide such discrimination with a singlemeasurement.

Following is a description of a method for selecting efficient mirrorsettings to achieve discrimination using a minimum number ofmeasurements. In matrix terminology, the task is to determine a minimumset of orthogonal vectors.

In accordance with the present invention, to this end one can use theWalsh-Hadamard Wavelet packets library. As known, these are richcollections of ±1, 0 patterns which will be used as elementary analysispatterns for discrimination. They are generated recursively as follows:(a) first, double the size of the pattern w in two ways either as (w,w)or as (w,−w). It is clear that if various n patterns wi of length n areorthogonal, then the 2n patterns of length 2n are also orthogonal. Thisis the simplest way to generate Hadamard-Walsh matrices.

The wavelet packet library consists of all sequences of length N havingbroken up in 2^(m) blocks, all except one are 0 and one block is filledwith a Walsh pattern (of ±1) of length 2^(l) where l+m=n. As known, aWalsh packet is a localized Walsh string of ±1. FIG. 28 illustrates all24 library elements for N=8.

A correlation of a vector x with a Walsh packet measures a variabilityof x at the location where the packet oscillates. The Walsh packetlibrary is a simple and computationally efficient analytic tool allowingsophisticated discrimination with simple binary operations. It can benoted that in fact, it is precisely the analog of the windowed Fouriertransform for binary arithmetic.

As an illustration, imagine two compounds A and B with subtledifferences in their spectrum. The task is to discriminate among them ina noisy environment and design efficient mirror configurations for DMAspectroscope. In accordance with a preferred embodiment, the followingprocedure can be used:

(1) Collect samples for both A and B, the number of samples collectedshould be representative of the inherent variability of themeasurements. A sample in this context is a full set x of the spectrumof the compound.

(2) Compute the inner product (x, w) for all samples X of A and (y, w)for all samples Y of B for each fixed Walsh product w.

(3) Measure the discrimination power pw of the pattern w to distinguishbetween compound A and B. This could be done by comparing thedistribution of the numbers {(x·w)} to the distribution of the numbers{(y, w)}, where the farther apart these distributions, the better theycan be distinguished.

(4) Select an orthogonal basis of patterns w maximizing the totaldiscrimination power and order them in decreasing order.

(5) Pick the top few patterns as an input to a multidimensionaldiscrimination method.

As an additional optional step in the above procedure, experiments canbe run using data on which to top few selected patterns failed, andrepeat steps 3, 4 and 5.

Because of the recursive structure of the W-packet library, it ispossible to achieve 2+3+4 in Nlog2 N computations per sample vector oflength N, i.e. essentially at the rate data collection. It should benoted that this procedure of basis selection for discrimination can alsobe used to enhance a variety of other signal processing tasks, such asdata compression, empirical regression and prediction, adaptive filterdesign and others. It allows to define a simple orthogonal transforminto more useful representations of the raw data. Further examples areconsidered below and illustrated in Section IV in the wheat proteinexample.

In this Section we considered the use of Hadamard processing to providesimple, computationally efficient and robust signal processing. Inaccordance with the present invention, the concept of using multiplesensors and/or detectors can be generalized to what is known ashyperspectral processing.

As known, current spectroscopic devices can be defined broadly into twocategories - point spectroscopy and hyperspectral imaging. Pointspectroscopy in general involves a single sensor measuring theelectromagnetic spectrum of a single sample (spatial point). Thismeasurement is repeated to provide a point-by-point scan of a scene ofinterest. In contrast, hyperspectral imaging generally uses an array ofsensors and associated detectors. Each sensor corresponds to the pixellocations of an image and measures a multitude of spectral bands. Theobjective of this imaging is to obtain a sequence of images, one foreach spectral band. At present, true hyperspectral imaging devices,having the ability to collect and process the full combination ofspectral and spatial data are not really practical as they requiresignificant storage space and computational power.

In accordance with the present invention, significant improvement overthe prior art can be achieved using hyperspectral processing thatfocuses of predefined characteristics of the data. For example, in manycases only a few particular spectral lines or bands out of the wholedata space are required to discriminate one substance over another. Itis also often the case that target samples do not posses very strong orsharp spectral lines, so it may not be necessary to use strong or sharpbands in the detection process. A selection of relatively broad bandsmay be sufficient do discriminate between the target object and thebackground. It should be apparent that the ease with which differentspatio-spectral bands can be selected and processed in accordance withthe present invention is ideally suited for such hyperspectrumapplications. A generalized block diagram of hyperspectral processing inaccordance with the invention is shown in FIG. 29. FIG. 30 illustratestwo spectral components (red and green) of a data cube produced byimaging the same object in different spectral bands. It is quite clearthat different images contain completely different kinds of informationabout the object. In one embodiment, using hyperspectral imaging from anairborne camera, one can identify different crops in a scene, based onthe predominant spectral characteristic of the crop. In anotherembodiment, one can use hyperspectral image of human skin with spectrumprogressing with increasing wavelength.

FIGS. 31A- 31E illustrate different embodiments of an imagingspectrograph in de-dispersive mode, that can be used in accordance withthis invention for hyperspectral imaging in the UV, visual, nearinfrared and infrared portions of the spectrum. For illustrationpurposes, the figures show a fiber optic probe head with a fixed numberof optical fibers. As shown, the fiber optic is placed at an exit slit.It will be apparent that a multitude of fiber optic elements anddetectors can be used in alternate embodiments.

FIG. 32 shows an axial and cross-sectional view of the fiber opticassembly illustrated in FIGS. 31A-31E.

FIG. 33 shows a physical arrangement of the fiber optic cable, detectorand the slit.

FIG. 34 illustrates a fiber optic surface contact probe head abuttingtissue to be examined;

FIGS. 35A and 35B illustrate a fiber optic e-Probe for pierced ears thatcan be used for medical monitoring applications in accordance with thepresent invention.

FIG. 36A, 36B and 36C illustrate different configurations of ahyperspectral adaptive wavelength advanced illuminating imagingspectrograph (HAWAIIS).

In FIG. 36A, DMD (shown illuminating the −1 order) is a programmablespatial light modulator that is used to select spatio/spectralcomponents falling upon and projecting from the combined entrance/exitslit. The illumination is fully programmable and can be modulated by anycontiguous or non-contiguous combination at up to 50 KHz. Thecorresponding spatial resolution element located at the Object/sample isthus illuminated and is simultaneously spectrally imaged by the CCD(located in order +1 with efficiency at 80%) as in typical CCD imagingspectrographs used for Raman spectral imaging.

With reference to FIGS. 36, the output of a broadband light source suchas a TQH light bulb(1001) is collected by a collection optic (lens 1002)and directed to a spatial light modulator such as the DMA used in thisexample(1003). Specific spatial resolution elements are selected bycomputer controlled DMA driver to propagate to the transmissiondiffraction grating(1005) via optic (lens 1004). The DMA(1003) shownilluminating the −1 order of the transmission diffraction grating(1005)is a programmable spatial light modulator that is used to selectspatio/spectral resolution elements projecting through the entrance/exitslit(#1007) collected and focused upon the sample(1009) by optic (lens1008). The spatio/spectral resolution elements illuminating the sampleare fully programmable. The sample is thus illuminated with specific andknown spectral resolution elements. The reflected spectral resolutionelements from specific spatial coordinates at the sample plane are thencollected and focused back through the entrance/exit slit by optic (lens1008). Optic (lens 1006) collimates the returned energy and presents itto the transmission diffraction grating(1005). The light is thendiffracted preferentially into the +1 order and is subsequentlycollected and focused by the optic (lens 1010) onto a 2D dectorarray(1011). This conjugate spectral imaging device has the advantage ofrejecting out of focus photons from the sample. Spectral resolutionelements absorbed or reflected are measured with spatial specificity bythe device.

FIGS. 43-47(A-D) illustrate hyperspectrum processing in accordance withthe present invention, including data maps, encodement mask, DMAprogrammable resolution using different numbers of mirrors and severalencodegrams.

D. Spatio-Spectral Tagging

One of the most important aspects of the present invention is the use ofmodulation of single array elements or groups of array elements to “tag”radiation impinging on these elements with its own pattern ofmodulation. In essence, this aspect of the invention allows to combinedata from a large number of array elements into a few processingchannels, possibly a single channel, without losing the identity of thesource and/or the spatial or spectral distribution of the data.

As known in the art, combination of different processing channels into asmaller number of channels is done using signal multiplexing. Inaccordance with the present invention, multiplexing of radiationcomponents which have been “tagged” or in some way encoded to retain theidentity of their source, is critical in various processing tasks, andin particular enables simple, robust implementations of practicaldevices. Thus, for example, in accordance with the principles of thepresent invention, using a micro mirror array, an optical router, anon-off switch (such as an LCD screen), enables simplified and robustimage formation with a single detector and further makes possibleincreasing the resolution of a small array of sensors to any desiredsize, as discussed in Section IV next.

The important point in this respect is that in accordance with thisinvention, methods for digitally-controlled modulation of sensor arraysare used to perform signal processing tasks while collecting data. Thus,the combination and binning of a plurality of radiation sources ismanipulated in accordance with this invention to perform calculations onthe analog data, which is traditionally done in the digital dataanalysis process. As a result, a whole processing step can be eliminatedby preselecting the switching modulation to perform the processingbefore the A/D conversion, thereby only converting data quantities ofinterest. This aspect of the present invention enables realtimerepresentation of the final processed data, which in processing-intenseapplications can be critical.

E. Data Compression, Feature Extraction and Diagnostics

By modulating the SLM array used in accordance with this invention, soas to compute inner products with elements of an orthogonal basis, theraw data can be converted directly on the sensor to provide the data intransform coordinates, such as Fourier transform, Wavelet transform,Hadamard, and others. This is in fact a key aspect of the resentinvention, and the reason why it is important is that the amount of datacollected is so large that it may swamp the processor or result ininsufficient bandwidth for storage and transmission. As known in theart, without some compression many imaging devices may become useless.As noted above, for hyperspectral imaging a full spectrum (a few hundreddata points) is collected for each individual pixel resulting in a dataglut. Thus, compression and feature extraction are essential to enable ameaningful image display. It will be appreciated that the resulting datafile is typically much smaller, providing significant savings in bothstorage and processing requirements. A simple example is the block 8×8Walsh expansion, which is automatically computed by appropriate mirrormodulation, the data measured is the actual compressed parameters.

In another related aspect of the present invention, data compression canalso be achieved by building an orthogonal basis of functions retainingthe important features for the task at hand. In a preferred embodiment,this can be achieved by use of the best basis algorithm. See, forexample, Coifman, R. R. and Wickerhauser, M. V., “Entropy-basedAlgorithms for Best Basis Selection”, IEEE Trans. Info. Theory 38(1992), 713-718, and U.S. Pat Nos. 5,526,299 and 5,384,725 to one of theinventors of this application. The referenced patents and publicationsare incorporated herein by reference.

By means of background, it is known that the reduction of dimensionalityof a set of data vectors can be accomplished using the projection ofsuch a set of vectors onto a orthogonal set of functions, which arelocalized in time and frequency. In a preferred embodiment, theprojections are defined as correlation of the data vectors with the setof discretized re-scaled Walsh functions, but any set of appropriatefunctions can be used instead, if necessary.

The best basis algorithm to one of the co-inventors of this applicationprovides a fast selection of an adapted representation for a signalchosen from a large library of orthonormal bases. Examples of suchlibraries are the local trigonometric bases and wavelet packet bases,both of which consist of waveforms localized in time and frequency. Anorthonormal basis in this setting corresponds to a tiling of thetime-frequency plane by rectangles of area one, but an arbitrary suchtiling in general does not correspond to an orthonormal basis. Only inthe case of the Haar wavelet packets is there a basis for every tiling,and a fast algorithm to find that basis is known. See, Thiele, C. andVillemoes. L., “A Fast Algorithm for Adapted Time-Frequency Tilings”,Applied and Computational Harmonic Analysis 3 (1996), 91-99, which isincorporated by reference.

Walsh packet analysis is a robust, fast, adaptable, and accuratealternative to traditional chemometric practice. Selection of featuresfor regression via this method reduces the problems of instabilityinherent in standard methods, and provides a means for simultaneouslyoptimizing and automating model calibration.

The Walsh system

W_(n)

=0 ^(∞)is defined recursively byW _(2n)(t)=W _(n)(2t)+(−1)^(n) W _(n)(2t−1)W _(2n+1)(t)=W _(n)(2t)−(−1)^(n) W _(n)(2t−1)

With W₀(t)=1 on 0≦t<1. If [0,1[×[0,∞[ is the time frequency plane,dyadic rectangles are subsets of the formI×ω=[2^(−j) k,2^(−j)(k+1)]×[2^(m) n,2^(m)(n+1)]

with j, k, m and n non-negative integers, and the tiles are therectangles of area one (j=m). A tile p is associated with a rescaledWalsh function by the expressionw _(p)(t)=2^(j/2) W _(n)(2^(j) t−k)

Fact: The function w_(p) and w_(q) are orthogonal if and only if thetiles p and q are disjoint. Thus, any disjoint tiling will give rise toan orthonormal basis of L²(0,1) consisting of rescaled Walsh functions.For any tiling B, we may represent a function f as$f = {\sum\limits_{p \in B}{\left\langle {f,w_{p}} \right\rangle w_{p}}}$

and may find an optimal such representation for a given additive costfunctional by choosing a tiling minimizing the cost evaluated on theexpansion coefficients.

In Section IV we consider an example contrasting the use of adaptiveWalsh packet methods with standard chemometrics for determining proteinconcentration in wheat. The data consists of two groups of wheatspectra, a calibration set with 50 samples and a validation set of 54samples. Each individual spectrum is given in units of log(1/R) where Ris the reflectance and is measured at 1011 wavelengths, uniformly spacedfrom 1001 nm to 2617 nm. Standard chemometric practice involvescomputing derivative-like quantities at some or all wavelengths andbuilding a calibration model from this data using least squares orpartial least squares regression.

To illustrate this, let Y_(i) be the percent protein for the i-thcalibration spectrum S_(i), and define the feature X_(i) to be$X_{i} = \frac{{S_{i}\left( {2182\quad{nm}} \right)} - {S_{i}\left( {2134\quad{nm}} \right)}}{{S_{i}\left( {2183\quad{nm}} \right)} - {S_{i}\left( {2260\quad{nm}} \right)}}$

where S_(i)(WLnm) is log(1/R) for the i-th spectrum at wavelength WL innanometers. This feature makes use of 4 of the 1011 pieces of spectraldata, and may be considered an approximate ratio of derivatives. Leastsquares provides a linear model AX_(i)+B yielding a prediction Ŷ_(i) ofY_(i). An estimate of the average percentage regression error is givenby:$\frac{100}{N}{\sum\limits_{i = 1}^{N}\frac{{\hat{Y_{i}} - Y_{i}}}{Y_{i}}}$

with N being the number of sample spectra in the given data set (N is 50for the calibration set). Retaining the same notation as for thecalibration set, one can compute the feature X_(i) for each validationspectrum S_(i) and use the above model to predict Y_(i) for thevalidation spectra. The average percentage regression error on thevalidation set is 0.62%, and this serves as the measure of success forthe model. This model is known to be state-of-the-art in terms of bothconcept and performance for this data, and will be used as point ofcomparison.

The wavelength-by-wavelength data of each spectrum is a presentation ofthe data in a particular coordinate system. Walsh packet analysisprovides a wealth of alternative coordinate systems in which to view thedata. In such a coordinate system, the coordinates of an individualspectrum would be the correlation of the spectrum with a given Walshpacket. The Walsh packets themselves are functions taking on the values1, −1, and 0 in particular patterns, providing a square-wave analogue oflocal sine and cosine expansions. Examples of Walsh packets are shown inFIG. 28.

In accordance with the present invention, such functions may be groupedtogether to form independent coordinate systems in different ways. Inparticular, the Walsh packet construction is dyadic in nature and yieldsfunctions having N=2^(k) sample values. For N=1024, the closest value ofN for the example case of spectra having 1011 sample values, the numberof different coordinate systems is approximately 10²⁷². If eachindividual Walsh packet is assigned a numeric cost (with somerestrictions), a fast search algorithm exists, which will find thecoordinate system of minimal (summed) cost out of all possible Walshcoordinate systems. Despite the large range for the search, thealgorithm is in not approximate, and provides a powerful tool forfinding representations adapted to specific tasks.

These ideas may be applied to the case of regression for the wheat datain question. Any Walsh packet provides a feature, not unlike the X_(i)computed above, simply by correlating the Walsh packet with each of thespectra. These correlations may be used to perform a linear regressionto predict the protein concentration. The regression error can be usedas a measure of the cost of the Walsh packet. A good coordinate systemfor performing regression is then one in which the cost, i.e. theregression error, is minimal. The fast algorithm mentioned above givesus the optimal such representation, and a regression model can bedeveloped out of the best K (by cost) of the coordinates selected.

In a particular embodiment, for each of the calibration spectra S_(i),first compute all possible Walsh packet features and then determine thelinear regression error in predicting the Y_(i) for each Walsh packet.Using this error as a cost measure, select a coordinate system optimizedfor regression, to provide a (sorted) set of features {X_(i)(1), . . . ,X_(i)(K)} associated with each spectrum S_(i). These features arecoordinates used to represent the original data, in the same way thatthe wavelength data itself does. Four features were used in the standardmodel described above, and, hence, one can choose K=4 and use partialleast squares regression to build a model for predicting Y_(i). Theaverage percentage regression error of this model on the validation dataset is 0.7%, and this decreases to 0.6% for K=10. FIG. 39A shows atypical wheat spectrum together with one of the top 4 Walsh packets usedin this model. The feature that is input to the regression model is thecorrelation of the Walsh packet with the wheat spectrum. (In this casethe Walsh feature computes a second derivative, which suppresses thebackground and detects the curvature of the hidden protein spectrum inthis region).

Similar performance is achieved by Walsh packet analysis using the samenumber of features. The benefit of using the latter becomes clear ifnoise is taken into account. Consider the following simple and naturalexperiment: add small amounts of Gaussian white noise to the spectra andrepeat the calibrations done above using both the standard model and theWalsh packet model. The results of this experiment are shown in FIG.41A, which plots the regression error versus the percentage noise energyfor both models (we show both the K=4 and the K=10 model for the Walshpacket case to emphasize their similarity). A very small amount of noisetakes the two models from being essentially equivalent to wildlydifferent, with the standard model having more than three times thepercentage error as the Walsh packet model. The source of thisinstability for the standard model is clear. The features used inbuilding the regression model are isolated wavelengths, and the additionof even a small amount of noise will perturb those featuressignificantly. The advantage of the Walsh packet model is clear in FIG.42. The feature being measured is a sum from many wavelengths, naturallyreducing the effect of the noise.

The Walsh packet method described here has other advantages as well. Oneof the most important is that of automation. The fast search algorithmautomatically selects the best Walsh packets for performing theregression. If the data set were changed to, say, blood samples andconcentrations of various analytes, the same algorithm would apply offthe shelf in determining optimal features. The standard model would needto start from scratch in determining via lengthy experiment whichwavelengths were most relevant.

Adaptability is also an important benefit. The optimality of thefeatures chosen is based on a numeric cost function, in this case alinear regression error. However, many cost functions may be used and ineach case a representation adapted to an associated task will be chosen.Optimal coordinates may be chosen for classification, compression,clustering, non-linear regression, and other tasks. In each case,automated feature selection chooses a robust set of new coordinatesadapted to the job in question.

IV. PRACTICAL APPLICATIONS

A number of applications of approaches and techniques used in accordancewith the present invention were discussed or pointed to in the abovedisclosure. In this Section we present several applications illustrativeof the advantages provided by the invention and the range of itspractical utility.

A. Gray Level Camera Processing System and Method

This application concerns a processing system, in which a video camerais synchronized to modulation of a tunable light source, allowinganalysis of the encoded spectral bands from a plurality of video imagesto provide a multispectral image. The utility of the application is duein part to the fact that it does not require special conditions—sincethe ambient light is not modulated it can be separated from the desiredspectral information. The system is the functional equivalent of imagingthe scene a number of times with a multiplicity of color filters. Itallows the formation of any virtual photographic color filter with anyabsorption spectrum desired. A composite image combining any of thesespectral bands can be formed to achieve a variety of image analysis,filtering and enhancing effects.

For example, an object with characteristic spectral signature can behighlighted by building a virtual filter transparent to this signatureand not to others (which should be suppressed). In particular, forseeing the concentration of protein in a wheat grain pile (the examplediscussed below) it would be enough to illuminate with two differentcombination of bands in sequence and take the difference of the twoconsecutive images. More elaborate encodements may be necessary if morespectral combinations must be measured independently, but the generalprinciple remains.

In a different embodiment, an ordinary video camera used in accordancewith this invention is equipped with a synchronized tunable lightsource, so that odd fields are illuminated with a spectral signaturethat is modulated from odd field to odd field, while the even fields aremodulated with the complementary spectral signature so that the combinedeven/odd light is white. Such an illumination system allows ordinaryvideo imaging which after digital demodulation provides detailedspectral information on the scene with the same capabilities as a graylevel camera.

This illumination processing system can be used for machine vision fortracking objects and anywhere that specific real time spectralinformation is useful.

In another embodiment, a gray level camera can measure severalpreselected light bands using, for example, 16 bands by illuminating thescene consecutively by the 16 bands and measuring one band at a time. Abetter result in accordance with this invention can be obtained byselecting 16 modulations, one for each band, and illuminatingsimultaneously the scene with all 16 colors. The sequence of 16 framescan be used to demultiplex the images. The advantages of multiplexingwill be appreciated by those of skill in the art, and include: bettersignal to noise ratio, elimination of ambient light interference,tunability to sensor dynamic range constraints, and others.

A straightforward extension of this idea is the use of this approach formultiplexing a low resolution sensor array to obtain better imagequality. For example, a 4×4 array of mirrors with Hadamard coding coulddistribute a scene of 400×400 pixels on a CCD array of 100×100 pixelsresulting in an effective array with 16 times the number of CCD.Further, the error could be reduced by a factor of four over a rasterscan of 16 scenes.

B. Chemical Composition Measurements In accordance with the presentinvention by irradiating a sample of material with well-chosen bands ofradiation that are separately identifiable using modulation, one candirectly measure constituents in the material of interest. Thismeasurement, for example, could be of the protein quantity in a wheatpile, different chemical compounds in human blood, or others. It shouldbe apparent that there is no real limitation on the type of measurementsthat can be performed, although the sensors, detectors and otherspecific components of the device, or its spectrum range may differ.

In the following example we illustrate the measurement of protein inwheat, also discussed in Section III.E. above. The data consists of twogroups of wheat spectra, a calibration set with 50 samples and avalidation set of 54 samples.

With further reference to Section III.B, FIG. 37 shows a DMA search bysplitting the scene. The detection is achieved by combining all photonsfrom the scene into a single detector, then splitting the scene in partsto achieve good localization. In this example, one is looking for asignal with energy in the red and blue bands. Spectrometer with twodetectors, as shown in FIG. 27 can be used, so that the blue light goesto the top region of the DMA, while the red goes to the bottom.

First, the algorithm checks if it is present in the whole scene bycollecting all photons into the spectrometer, which looks for thepresence of the spectral energies. Once the particular spectrum band isdetected, the scene is split into four quarters and each is analyzed forpresence of target. The procedure continues until the target isdetected.

FIG. 38 illustrates the sum of wheat spectra training data (top), sum of|w| for top 10 wavelet packets (middle), and an example of proteinspectra—soy protein (bottom). The goal is to estimate the amount ofprotein present in wheat. The middle portion of the figure shows theregion where the Walsh packets provide useful parameters forchemo-metric estimation.

FIG. 39 illustrates the top 10 wavelet packets in local regression basisselected using 50 training samples. Each Walsh packet provides ameasurement useful for estimation. For example, the top line indicatesthat by combining the two narrow bands at the ends and the subtractingthe middle band we get a quantity that is linearly related to theprotein concentration. FIG. 40 is a scatter plot of protein content(test data) vs. correlation with top wavelet packet. This illustrates asimple mechanism to directly measure relative concentration of desiredingredients of a mixture using the present invention.

It will be appreciated that in this case one could use an LED-basedflashlight illuminating in the three bands with a modulated light, whichis then imaged with a CCD video camera that converts any group ofconsecutive three images into an image of protein concentration. Anotherimplementation is to replace the RGB filters on a video camera by threefilters corresponding to the protein bands, to be displayed aftersubstraction as false RGB. Various other alternative exist and will beappreciated by those of skill in the art.

FIG. 41 illustrates PLS regression of protein content of test data:using top 10 wavelet packets (in green −1.87% error, from 6 LVs) and top100 (in red −1.54% error from 2 LVs)—compare with error of 1.62% from 14LVs using all original data. This graph compares the performance of thesimple method described above to the true concentration values.

FIG. 42 illustrates the advantage of DNA-based Hadamard Spectroscopy interms of visible improvement in the SNR of the signal for the HadamardEncoding over the regular raster scan.

It will be appreciated that the above approach can be generalized to amethod of detecting a chemical compound with known absorption lines. Inparticular, a simple detection mechanism for compounds with knownabsorption is to use an active illumination system that transmitsradiation (such as light) only in areas of the absorption spectrum ofthe compound. The resulting reflected light will be weakest where thecompound is present, resulting in dark shadows in the image (afterprocessing away ambient light by, for example, subtracting the imagebefore illumination). Clearly, this approach can be used to dynamicallytrack objects in a video scene. For example, a red ball could be trackedin a video sequence having many other red objects, simply bycharacterizing the red signature of the ball, and tuning theillumination to it, or by processing the refined color discrimination.Clearly this capability is useful for interactive TV or video-gaming,machine vision, medical diagnostics, or other related applications.Naturally, similar processing can be applied in the infrared range (orUV) to be combined with infrared cameras to obtain a broad variety ofcolor night vision or (heat vision), tuned to specific imaging tasks. Toencode the received spatial radiation components one can use pulse codemodulation (PCM), pulse width modulation (PWM), time divisionmultiplexing (TDM) and any other modulation technique that has theproperty of identifying specific elements of a complex signal or image.

In accordance with the invention, in particular applications one canrapidly switch between the tuned light and its complement, arrangingthat the difference will display the analyte of interest with thehighest contrast. In addition, it is noted that the analyte of interestwill flicker, enabling detection by the eye. Applications of thisapproach in cancer detection in vivo, on operating table, can easily beforeseen.

C. Encoding Information in Physical Matter

A straightforward extension of the present invention is a method forinitiating select chemical reactions using a tunable light source. Inaccordance with this aspect of the invention, the tunable light sourceof this invention can be tuned to the absorption profile of a compoundthat is activated by absorbing energy to achieve, for example, curing,drying, heating, cooking of specific compounds in a mixture and otherdesired results. Applications further include photodynamic therapy, suchas used in jaundice treatment, chemotherapy, and others.

Yet another application is a method for conducting spectroscopy withdetermining the contribution of individual radiation components frommultiplexed measurements of encoded spatio-spectral components. Inparticular a multiplicity of coded light in the UV band could be used tocause fluorescence of biological materials, the fluorescent effect canbe analyzed to relate to the specific coded UV frequency allowing amultiplicity of measurements to occur in a multiplexed form. Anillumination spectrum can be designed to dynamically stimulate thematerial to produce a detectable characteristic signature, includingfluorescence effects and multiple fluorescent effects, as well a Ramanand polarization effects. Shining UV light in various selectedwavelengths is known to provoke characteristic fluorescence, which whenspectrally analyzed can be used to discriminate between variouscategories of living or dead cells.

Another important application of the system and method of this inventionis the use of the OSPU as a correlator or mask in an optical computationdevice. For example, an SLM, such as DMA can act as a spatial filter ormask placed at the focal length of a lens or set of lenses. Asillustrated above, the SLM can be configured to reject specific spatialresolution elements, so that the subsequent image has properties thatare consistent with spatial filtering in Fourier space. It will beapparent that the transform of the image by optical means is spatiallyeffected, and that the spatial resolution of images produced in thismanner can be altered in a desired way. Exactly how the spatialresolution is altered will depend on the particular application and neednot be considered in further detail.

Yet another area of use is performing certain signal processingfunctions in an analog domain. For example, spatial processing with aDMA can be achieved directly in order to acquire various combinations ofspatial patterns. Thus, an array of mirrors can be arranged to have allmirrors of the center of the image point to one detector, while all theperiphery may point to another. Another useful arrangement designed todetect vertical edges will raster scan a group of, for example, 2×2mirrors pointing left combined with an adjacent group of 2×2 mirrorspointing right. This corresponds to a convolution of the image with anedge detector. The ability to design filters made out of patterns of0,1,−1 i.e., mirror configurations, will enable the imaging device toonly measure those features which are most useful for display,discrimination or identification of spatial patterns.

The design of filters can be done empirically by using the automaticbest basis algorithms for discrimination, discussed above, which isachieved by collecting data for a class of objects needing detection,and processing all filters in the Walsh Hadamard Library of waveletpackets for optimal discrimination value. The offline default filterscan then be upgraded online in realtime to adapt to filed conditions andlocal clutter and interferences.

While the foregoing has described and illustrated aspects of variousembodiments of the present invention, those skilled in the art willrecognize that alternative components and techniques, and/orcombinations and permutations of the described components andtechniques, can be substituted for, or added to, the embodimentsdescribed herein. It is intended, therefore, that the present inventionnot be defined by the specific embodiments described herein, but ratherby the appended claims, which are intended to be construed in accordancewith the well-settled principles of claim construction, including that:each claim should be given its broadest reasonable interpretationconsistent with the specification; limitations should not be read fromthe specification or drawings into the claims; words in a claim shouldbe given their plain, ordinary, and generic meaning, unless it isreadily apparent from the specification that an unusual meaning wasintended; an absence of the specific words “means for” connotesapplicants’ intent not to invoke 35 U.S.C. §112 (6) in construing thelimitation; where the phrase “means for” precedes a data processing ormanipulation “function,” it is intended that the resultingmeans-plus-function element be construed to cover any, and all, computerimplementation(s) of the recited “function”; a claim that contains morethan one computer-implemented means-plus-function element should not beconstrued to require that each means-plus-function element must be astructurally distinct entity (such as a particular piece of hardware orblock of code); rather, such claim should be construed merely to requirethat the overall combination of hardware/firmware/software whichimplements the invention must, as a whole, implement at least thefunction(s) called for by the claim's means-plus-function element(s).

1-9. (canceled)
 10. A spectral filter comprising: a mirror array; adispersion device for dispersing a range of wavelengths of radiation;and a controller for controlling said mirror array so that said spectralfilter passes or rejects the radiation from any arbitrary combination ofelements of said mirror array; and wherein said mirror array and saiddispersion device are disposed so that different bands of saidwavelengths of radiation are reflected upon corresponding separatecolumns or rows of said mirror array.
 11. The spectral filter of claim10, wherein said mirror array is disposed so that different spatialresolution elements are reflected upon corresponding separate rows orcolumns of said mirror array.
 12. The spectral filter of claim 11,wherein said different spatial resolution elements are reflected uponcorresponding separate rows if said different bands of said wavelengthsare reflected upon corresponding separate columns; and wherein saiddifferent spatial resolution elements are reflected upon correspondingseparate columns if said different bands of said wavelengths arereflected upon corresponding separate rows.
 13. The spectral filter ofclaim 12, wherein said controller is disposed to control said mirrorarray so that the array passes or rejects any arbitrary combination ofsaid bands of said wavelengths of radiation and any arbitrarycombination of said spatial resolution elements.
 14. A spectralmeasurement system for measuring compounds of interest comprising thespectral filter of claim 13 and a detector; and wherein said controlleris disposed to control said mirror array so that spectral resolutionelements of said mirror array corresponding to said compounds ofinterest and spatial resolution elements corresponding to said compoundsof interest are directed simultaneously to said detector formeasurement.
 15. The spectral filter of claim 10, wherein said mirrorarray is a 2D micro-mirror array spatial light modulator.
 16. A spectralfilter comprising: a mirror array comprised of a plurality of fixedmirror elements; a dispersion device for dispersing a range ofwavelengths of radiation; and wherein said mirror array and saiddispersion device are disposed so that different bands of saidwavelengths of radiation are reflected upon corresponding separatecolumns or rows of said mirror array so that said spectral filter passesor rejects the radiation from a predetermined combination of elements ofsaid mirror array.
 17. The spectral filter of claim 16, wherein theposition of at least some of said fixed mirror elements and the absenceof mirror elements in at least one other position, influences thewavelengths of light passed by said spectral filter.
 18. A spectrallytunable light source, comprising the spectral filter of claim 16 and abroadband radiation source; and wherein an input to said spectral filtercomprises radiation from said broadband radiation source; and whereinsaid spectrally tunable light source is operable to pass or reject theradiation form a predetermined combination of elements of said mirrorarray.
 19. The spectrally tunable light source of claim 18, wherein theposition of at least some of said fixed mirror elements and the absenceof mirror elements in at least one other position, influences thewavelengths of light output by said spectrally tunable light source. 20.A spectrally tunable light source, comprising: a broadband radiationsource; a mirror array; a dispersion device dispersing a range ofwavelengths of radiation; and a controller for controlling said mirrorarray so that the array passes or rejects the radiation from anyarbitrary combination of elements of said mirror array; and wherein saidmirror array and said dispersion device are disposed so that differentbands of said wavelengths of radiation are reflected upon correspondingseparate columns or rows of said mirror array.
 21. The spectrallytunable light source of claim 20, wherein said mirror array is disposedso that different spatial resolution elements are reflected uponcorresponding separate rows or columns of said array.
 22. The spectrallytunable light source of claim 21, wherein said different spatialresolution elements are reflected upon corresponding separate rows ifsaid different bands of said wavelengths are reflected uponcorresponding separate columns; and wherein said different spatialresolution elements are reflected upon corresponding separate columns ifsaid different bands of said wavelengths are reflected uponcorresponding separate rows.
 23. The spectrally tunable light source ofclaim 22, wherein said controller is disposed to control said mirrorarray so that the array passes or rejects any arbitrary combination ofsaid bands of said wavelengths of radiation and any arbitrarycombination of said spatial resolution elements.
 24. A spectralmeasurement system for measuring compounds of interest comprising thespectrally tunable light source of claim 23 and a detector; and whereinsaid controller is disposed to control said mirror array so thatspectral resolution elements of said mirror array corresponding to saidcompounds of interest and spatial resolution elements corresponding tosaid compounds of interest are directed simultaneously to said detectorfor measurement.
 25. The spectrally tunable light source of claim 20,wherein said mirror array is a 2D micro-mirror array spatial lightmodulator.